System and method of radiograph correction and visualization

ABSTRACT

Systems and methods of radiograph correction and visualization are disclosed. Certain embodiments provide a method for generating a 3D model of at least part of one anatomical object based on one or more radiographs. The method further includes positioning the 3D model based on information indicative of a normalized projection comprising information indicative of a desired position and orientation of the at least part of one anatomical object with respect to the projection plane. The method further includes generating a 2D projection of the 3D model onto the projection plane. The method further includes generating one or more modified radiographs of the at least part of one anatomical object based on the 2D projection.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/630,840, filed Jan. 13, 2020, which is a National Stage Entry ofInternational Application No. PCT/US2018/042073, filed Jul. 13, 2018,which claims the benefit of U.S. Provisional Patent No. 62/666,962,filed May 4, 2018, U.S. Provisional Patent No. 62/664,865, filed Apr.30, 2018, and U.S. Provisional Patent No. 62/532,794, filed Jul. 14,2017. The entire contents of each of these applications are incorporatedby reference herein in their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

This application relates to medical imaging. In some aspects, thisapplication relates specifically to systems and methods for radiographcorrection and visualization.

Description of the Related Technology

Radiographs are images created using radiography, which is an imagingtechnique using X-rays to view the internal form of an object. Forexample, an imaging device includes a source device that generatesX-rays that are projected toward an anatomical object (e.g., a bone in apatient). Some amount of the X-rays is absorbed by the anatomical objectand the remaining portion of the X-rays is captured by a detector of theimaging device placed behind the anatomical object relative to thesource device. The resulting radiograph is a two-dimensional (2D) image.In some cases, the radiograph has gray values (e.g., of pixels that formthe 2D image) indicative of the absorption amount of the various X-rays,and accordingly indicative of the density and structural composition ofthe anatomical object. In some cases, the radiograph has color values(e.g., of pixels that form the 2D image) indicative of the absorptionamount of the various X-rays, and accordingly indicative of the densityand structural composition of the anatomical object. The radiograph, insome cases, may have other values.

The use of radiographs to plan surgery, such as orthopedic surgery, iscommon. Typically, planning of such surgery of an anatomical objectincludes formulating a diagnosis of the condition of an anatomicalobject, taking measurements of the anatomical object and/or areassurrounding the anatomical object, planning osteotomies of theanatomical object and/or areas surrounding the anatomical object, 2Dtemplating (e.g., choosing an implant type, position, orientation,and/or size based on an overlay of a 2D outline of the implant onto aradiograph of the anatomical object), etc. Similarly radiographs orfluoroscopy images are used to perform intra-operative andpost-operative verification or measurements of the anatomical objectand/or areas surrounding the anatomical object.

One common problem is that a radiograph only contains a 2D projection ofthe anatomy (e.g., of the anatomical object), and that many kinds ofplanning activities assume a particular view of the anatomy (e.g.anteroposterior (AP), lateral, etc.). If the anatomy cannot be properlypositioned with respect to the imaging device used to take theradiograph (e.g., a patient is in pain or less mobile), it may not bepossible to obtain a radiograph in the particular view desired. Theparticular view can refer to a position of the anatomy with respect tothe imaging machine (e.g., source device and/or detector), and/or to theposition and orientation of individual anatomical objects of the anatomywith respect to each other (e.g. the internal/external rotation,flexion/extension and/or abduction/adduction of a joint). If theradiograph's 2D projection does not fully correspond to the particularview of the anatomy, it could lead to measurement errors or incorrect 2Dtemplating (e.g., as discussed in Lechler, Philipp; Frink, Michael;Gulati, Aashish; Murray, David; Renkawitz, Tobias; Bucking, Benjamin;Ruchholtz, Steffen; Boese, Christoph Kolja, “The influence of hiprotation on femoral offset in plain radiographs”, Acta orthopaedica,August 2014, Vol. 85(4), pp. 389-95). For example, a clinician mighthave to make mental corrections for the incorrect view, which may not beaccurate.

One solution to this problem is suggested by Tannast et al. (2008)(Tannast, Moritz; Mistry, Sapan; Steppacher, Simon D.; Reichenbach,Stephan; Langlotz, Frank; Siebenrock, Klaus A.; Zheng, Guoyan,“Radiographic analysis of femoroacetabular impingement with Hip 2norm-reliable and validated”, Journal of Orthopaedic Research, September2008, Vol. 26(9), pp. 1199-1205) for radiographic examination of thehuman pelvis. The software Hip2Norm uses knowledge of the radiographicprojection and the anatomy to produce, based on regular radiographs,line drawings of the pelvis and of the acetabular rim as they would beseen on a true AP radiograph, so as to allow standardized evaluation ofradiographic parameters for the description of acetabular morphology.

An alternative solution is suggested in WO 2011/098895 A2. Thispublication describes a method of registering and adjusting a3-dimensional multi-object statistical model to one or more standardradiographs in order to obtain a three-dimensional (3D) reconstructionof the anatomy for diagnosis purposes.

A main drawback with both such solutions is that they present thecorrected information about the patient's anatomy in a way with whichclinicians are unfamiliar. A 2D line drawing still requires theclinician mentally combining the shapes of the lines with the visualinformation of the radiograph, and a 3D model of the anatomy does noteasily allow clinicians to perform the diagnostic steps that they areused to performing on 2D radiographs.

Certain embodiments herein comprise a method that builds on and expandson principles from both publications to modify original regularradiographs to show the anatomy in a normalized position (e.g., desiredposition and orientation) and still contain the visual information usedfor diagnosis/planning that is present in the original unmodifiedradiographs.

SUMMARY

Certain embodiments provide a computer-implemented method of generatinga computer-based radiographic representation of at least part of oneanatomical object. The method includes obtaining, at a computing device,one or more radiographs of at least part of one anatomical object, eachof the one or more radiographs comprising a 2D visual representation ofthe at least part of one anatomical object in a projection plane. Themethod further includes obtaining, by the computing device, informationindicative of a normalized projection comprising information indicativeof a desired position and orientation of the at least part of oneanatomical object with respect to the projection plane. The methodfurther includes generating, by the computing device, a 3D model of theat least part of one anatomical object based on the one or moreradiographs. The method further includes positioning, by the computingdevice, the 3D model based on the information indicative of thenormalized projection. The method further includes generating, by thecomputing device, a 2D projection of the 3D model onto the projectionplane. The method further includes generating, by the computing device,one or more modified radiographs of the at least part of one anatomicalobject based on the 2D projection.

Certain embodiments provide a non-transitory computer-readable mediumhaving computer-executable instructions stored thereon, which, whenexecuted by a processor of a computing device, cause the computingdevice to perform the described method.

Certain embodiments provide a computing device comprising a memory and aprocessor configured to perform the described method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of one example of a computing environmentsuitable for practicing various embodiments disclosed herein.

FIG. 2 is a high level system diagram of a computing system that may beused in accordance with one or more embodiments.

FIGS. 3 and 3A illustrate a flow chart showing a process for generatinga computer-based radiographic representation of at least one anatomicalobject, according to certain embodiments.

FIG. 4 illustrates an example radiograph of a proximal femur and part ofa pelvis.

FIG. 5 illustrates an example of a 3D model of at least one anatomicalobject generated from the radiograph of FIG. 4 .

FIG. 6 illustrates an example of the 3D model of FIG. 5 repositioned.

FIGS. 7 and 8 illustrate an example of a 2D projection as a contour linerepresentation.

FIG. 9 illustrates an example of a 2D projection as a syntheticradiograph.

FIG. 10 illustrates an example of the radiograph of FIG. 4 with contoursof the at least one anatomical object identified.

FIG. 11 illustrates an example of the radiograph of FIG. 4 with bothcontours of FIG. 10 and contour representation of FIGS. 7 and 8 overlaidon the radiograph.

FIG. 12 illustrates an example of a modified radiograph that is a morphof the radiograph of FIG. 10 such that the contours of FIG. 10 alignwith the contour representation of FIGS. 7 and 8 .

FIG. 13 illustrates an example of an original synthetic radiograph whichis to be registered to a repositioned synthetic radiograph.

FIG. 14 illustrates an example of an original radiograph and a modifiedradiograph.

FIG. 15 illustrates an example of a modified radiograph.

FIG. 16 illustrates a modified AP pelvic radiograph with a measurementof a lateral center edge angle.

DETAILED DESCRIPTION OF CERTAIN INVENTIVE EMBODIMENTS

Certain embodiments herein provide systems and methods for generating acomputer-based radiographic representation of at least one anatomicalobject. For example, the anatomical object may include one or moreanatomy/anatomical parts (e.g., hip, knee, shoulder, etc.) of a human,such as one or more bones corresponding to or associated with one ormore anatomy parts of a human. In particular, in certain embodiments thecomputer-based radiographic representation is generated based on one ormore actual radiographs containing visual information of the anatomicalobject, and a 3D model of the anatomical object generated based on theone or more actual radiographs. It should be noted that in certainembodiments, one or more of the methods described herein is acomputer-implemented method. For example, computer-based imagesnecessarily need to be generated on a computing device and thegeneration of such images is a computer centric problem. In particular,such images cannot be generated mentally or by a human alone without acomputing device. Further, certain steps may be performed automatically,manually by a user of a computing device, or partially manually andpartially automatically such as based on input from a user of acomputing device. Further, in certain embodiments, a user may be aperson, such as a clinician, engineer, technician, medical professional,etc., that may use a computing device to, or the computing device itselfmay automatically perform one or more steps of one or more methodsdescribed herein.

Embodiments described herein provide a technical solution to a technicalproblem. In particular, as discussed, due to the limitations oftraditional medical imaging, it is not always feasible to takeradiographs of anatomical objects in different orientations.Accordingly, radiographic images of anatomical objects are not able tobe produced in certain orientations needed to plan medical procedures.Therefore, present medical imaging techniques and medical proceduressuffer from the technical problem of not being able to produceradiographic images in the needed orientations. Certain embodimentsprovided herein provide a technical solution to this technical problem.In particular certain embodiments provide specific techniques togenerate radiographs for visualization (e.g., on computing systems) ofanatomical objects repositioned in the radiographs to differentorientations than the original radiograph. The generated radiographsbeneficially provide features of the original radiograph accuratelyrepositioned in the generated radiograph, thereby improving thetechnical field of medical imaging.

The systems and methods described herein may be implemented in acomputing environment comprising one or more computing devicesconfigured to provide various functionalities. FIG. 1 is an example of acomputer environment 100 suitable for implementing certain embodimentsdescribed herein. The computer environment 100 may include a network101. The network 101 may take various forms. For example, the network101 may be a local area network installed at a surgical site. In someembodiments, the network 101 may be a wide area network such as theInternet. In other embodiments, the network 101 may be a combination oflocal area networks and wide area networks. Typically, the network willallow for secured communications and data to be shared between variouscomputing devices. Among these computing devices are a client device104. The client device 104 may be a typical personal computer devicethat runs an off-the-shelf operating systems such as Windows, Mac OS,Linux, Chrome OS, or some other operating system. The client device 104may have application software installed to allow it to interact via thenetwork 101 with other software stored on various other modules anddevices in the computing environment 100. This application software maytake the form of a web browser capable of accessing a remote applicationservice. Alternatively, the application software may be a clientapplication installed in the operating system of the client device 104.Client device 104 may also take the form of a specialized computer,specifically designed for medical imaging work, or even morespecifically for generating a computer-based radiographic representationof at least one anatomical object. The client device 104 may furthertake the form of a mobile device or tablet computer configured tocommunicate via the network 101 and further configured to run one ormore software modules to allow a user to perform various methodsdescribed herein.

The computer environment 100 may further include image data storage 106.Typically, the image data storage 106 takes the form of a databasedesigned to store image files captured by a scanning device 111 (e.g.,X-ray imaging device). These images may be Digital Imaging andCommunications in Medicine (DICOM) images, or other types of images. Theimage data storage 106 may be part of a scanning device 111, oralternatively it may be part of a client computing device 104. The imagedata storage 106 may also be in a standalone database having dedicatedstorage optimized for medical image data. The image data store 106 mayfurther include in the same database or a separate database 2-D and or3-D digital representations/images of designs implants, as furtherdiscussed herein. The computer environment 100 may also include ascanning device 111. The scanning device 111 may typically be a medicalimaging device which scans/images a patient to create images of theiranatomy. In the computing environment 100 shown in FIG. 1 , the scanningdevice 111 may be an X-ray device that generates 2D radiographs, asdiscussed.

As will be explained in detail below, the scanning device 111 may beconfigured to create 2D radiographic images, referred to as radiographs,of anatomical objects. Those images may be stored in the image datastorage 106, and utilized to create 3D models of the anatomical objects.The 3D models are further used to generate modified radiographs of theanatomical objects including the anatomical objects in differentpositions than the original radiographs. To that end, the computingenvironment 100 may also include an image processing module 108. Theimage processing module 108 may take the form of computer software,hardware, or a combination of both which retrieves the medical imagingdata from image data storage 106 and generates 3D models and modifiedradiographs as further discussed herein. In some embodiments, the imageprocessing module 108 may be provided via a web-based networkapplication that is accessed by a computer over the network (such asclient device 104, for example). Alternatively, the image processingmodule may be a software application that is installed directly on theclient device 104, and accesses image data storage 106 via the network101. In general, the image processing module 108 may be any combinationof software and/or hardware located within the computing environment 100which provides image processing capabilities on the image data storedwithin the image data storage 106.

The computing environment also may include a measurement and analysismodule 110 (“measurement and analysis module”). The measurement andanalysis module 110 may be software that is complementary to and/orbundled with the image processing module 108. The measurement andanalysis module may be an application configured to determinemeasurements of anatomical objects, such as in 3D models of theanatomical objects such as those generated according to the techniquesdiscussed herein. As with the image processing module 108, themeasurement and analysis module 110 may be a network-based applicationwhich is accessed via a web browser by one or more client devices 104.It may also be a native application installed into the operating systemof a computer such as, client device 104 for example. In still otherembodiments, the measurement and analysis module 110 may be a networkapplication which is run as a client/server implementation.

The computing environment also may include a visualization module 112.The visualization module 112 may be software that is complementary toand/or bundled with the image processing module 108. The visualizationmodule 112 may be an application configured to provide differentvisualizations of anatomical objects. For example, visualization module112 may cause one or more 3D models and/or or radiographs (e.g.,modified or original) to be displayed on a display of a computingdevice, such as client device 104, by rendering images for display.Visualization module 112, as will be discussed, may render images withdifferent colors, sizes, according to different user interfaces, etc.for display. Visualization module 112 may further render images overlaidon top of other images, such as images or renders (e.g., 2D or 3D) ofimplants on anatomical objects, as further discussed herein. As with theimage processing module 108, the visualization module 112 may be anetwork-based application which is accessed via a web browser by one ormore client devices 104. It may also be a native application installedinto the operating system of a computer such as, client device 104 forexample. In still other embodiments, the visualization module 112 may bea network application which is run as a client/server implementation.

Various embodiments of the invention may be implemented using generaland/or special purpose computing devices. Turning now to FIG. 2 , anexample of a computing device 200 suitable for implementing variousembodiments of the invention is shown. The computer system 200 maygenerally take the form of computer hardware configured to executecertain processes and instructions in accordance with various aspects ofone or more embodiments described herein. The computer hardware may be asingle computer or it may be multiple computers configured to worktogether. The computing device 200 includes a processor 202. Theprocessor 202 may be one or more standard personal computer processorsuch as those designed and/or distributed by Intel, Advanced MicroDevices, Apple, or ARM. The processor 202 may also be a more specializedprocessor designed specifically for image processing and/or analysis.The computing device 200 may also include a display 204. The display 204may be a standard computer monitor, such as an LCD monitor as is wellknown. The display 204 may also take the form of a display integratedinto the body of the computing device, for example as with an all-in-onecomputing device or a tablet computer.

The computing device 200 may also include input/output devices 206.These may include standard peripherals such as keyboards, mice,printers, and other basic I/O software and hardware. The computingdevice 200 may further include memory 208. The memory 208 may takevarious forms. For example, the memory 208 may include volatile memory210. The volatile memory 210 may be some form of random access memory,and may be generally configured to load executable software modules intomemory so that the software modules may be executed by the processor 202in a manner well known in the art. The software modules may be stored ina nonvolatile memory 212. The non-volatile memory 212 may take the formof a hard disk drive, a flash memory, a solid state hard drive or someother form of non-volatile memory. The non-volatile memory 212 may alsobe used to store non-executable data, such database files and the like.

The computer device 200 also may include a network interface 214. Thenetwork interface may take the form of a network interface card and itscorresponding software drivers and/or firmware configured to provide thesystem 200 with access to a network (such as the Internet, for example).The network interface card 214 may be configured to access variousdifferent types of networks, such as those described above in connectionwith FIG. 1 . For example the network interface card 214 may beconfigured to access private networks that are not publicly accessible.The network interface card 214 may also be configured to access wirelessnetworks such using wireless data transfer technologies such as EVDO,WiMax, or LTE network. Although a single network interface 214 is shownin FIG. 2 , multiple network interface cards 214 may be present in orderto access different types of networks. In addition, a single networkinterface card 214 may be configured to allow access to multipledifferent types of networks.

In general, the computing environment 200 shown in FIG. 2 may generallyinclude one, a few, or many different types of computing devices 200which work together to carry out various embodiments described below. Askilled artisan will readily appreciate that various different types ofcomputing devices and network configurations may be implemented to carryout the inventive systems and methods disclosed herein.

FIG. 3 illustrates a flow chart showing a process 300 for generating acomputer-based radiographic representation of at least one anatomicalobject according to certain embodiments. It should be noted that incertain embodiments, process 300 is a computer-implemented process.Further, certain blocks may be performed automatically, manually by auser of a computing device, or partially manually and partiallyautomatically such as based on input from a user of a computing device.

Process 300 begins at block 302, wherein one or more radiographs of atleast one anatomical object are obtained. In certain aspects, the one ormore radiographs may be for at least part of the at least one anatomicalobject and the techniques described herein may be similarly applied toat least part of at least one anatomical object and not necessarilycomplete anatomical objects. The radiographs may be acquired using thescanning device 111 shown in FIG. 1 , such as an X-ray scanner. Eachradiograph includes a 2D visual representation of the at least oneanatomical object in a projection plane. In certain embodiments, theprojection plane corresponds to a plane of a detector of the X-rayscanner. The radiograph (or radiographs) acquired using the scanningdevice 111 may be stored in image data storage 106 or some othercomputer memory accessible via the computer network 101. Obtaining theradiographs may refer to acquiring the radiographs using the X-rayscanner, and/or retrieving the radiographs from memory. FIG. 4illustrates an example radiograph 400 of a proximal femur and part of apelvis. As shown, the AP view of the femur is not clear in radiograph400. In particular, the femur is rotated, flexed, and adducted.

In certain embodiments, the radiographs can be regular 2D radiographsobtained by projecting a beam of X-rays from a source through theanatomy of a patient onto a detector. The radiographs may contain visualinformation of one or more anatomical objects corresponding to one ormore anatomy parts of interest. In certain embodiments, the radiographsinclude information from a locating/scaling device to determine thescaling factor and/or determine the position and/or orientation of thepatient with respect to the X-ray detector plane. This information canbe used to determine the position of the source and/or an anatomicalobject with respect to the projection plane (e.g., corresponding to thedetector). Alternatively such information can be obtained separately.

The process then moves to block 304. At block 304, informationindicative of a normalized projection (e.g., a definition of anormalized projection) is obtained for the at least one anatomicalobject. In certain embodiments, the information indicative of anormalized projection comprises information indicative of a desiredposition and orientation of the at least one anatomical object relativeto the projection plane. In certain embodiments, the informationindicative of a normalized projection further comprises informationindicative of the source (e.g., information indicative of the locationof the source) of the X-rays used to generate the radiographs withrespect to the projection plane (e.g., information indicative of theprojection source (e.g., location of the projection source) of the X-raymachine). In certain embodiments, the position and orientation of ananatomical object can be defined in terms of its anatomical coordinatesystem. The information indicative of a normalized projection may bestored on and obtained from a memory, such as image data storage 106 oranother memory. The information indicative of a normalized projectionmay be generated by a user, such as using a computing device, such asclient device 104 or another computing device.

The process continues to block 306. At block 306, a 3D model of the atleast one anatomical object is generated based on the one or moreradiographs. For example, image processing module 108 generates the 3Dmodel (also referred to as a virtual 3D model). In certain embodiments,the 3D model is generated from a deformable 3D model includingstatistical information (e.g., one or more of statistical shape model,articulated statistical shape model, active shape model, appearancemodel, intensity model, etc.) about the at least one anatomical object.In certain embodiments, the 3D model is generated by registering andadjusting a shape of the deformable 3D model to align with a shape ofthe at least one anatomical object in the one or more radiographsaccording to the visual information (e.g., 2D visual representation ofthe at least one anatomical object) in the one or more radiographs. Incertain embodiments, the 3D model is generated accordingly to knownmethods in the art, such as those described in Balestra et al. (2014)(S. Balestra, S. Schumann, J. Heverhagen, L. Nolte, and G. Zheng,“Articulated Statistical Shape Model-Based 2D-3D Reconstruction of a HipJoint”, in: Information Processing in Computer-Assisted Interventions,Proceedings of IPCAI 2014, June 2014, pp. 128-137). In certainembodiments, the 3D model is at least one virtual 3D model of the atleast one anatomical object. For example, the 3D model may compriseseparate virtual 3D models for each of the anatomical objects.

FIG. 5 illustrates an example of a 3D model 500 of at least oneanatomical object generated from radiograph 400 of FIG. 4 . Inparticular, the 3D model 500 corresponds to the part of the pelvis andfemur from FIG. 4 . As shown, in 3D model 500, the pelvis is correctlyoriented with respect to a projection plane of the radiograph 400 for anAP view. However, in 3D model 500, the femur is internally rotated,slightly adducted, and in flexion.

Continuing at block 308, a 2D projection of the 3D model of the at leastone anatomical object is generated. For example, image processing module108 generates the 2D projection. In certain aspects, as furtherdiscussed herein, the 2D projection can be a contour line representationand/or synthetic radiograph. In certain embodiments, block 308 comprisesblocks 310 and 312, shown in FIG. 3A.

At block 310, the 3D model (e.g., one or more virtual 3D models) isrepositioned (e.g., with respect to the projection plane, an optimalplane, other objects/anatomy, etc.). For example, the 3D model isrepositioned automatically by, or by a user using, image processingmodule 108. In certain embodiments, the 3D model is also repositionedwith respect to the source of the X-rays. The 3D model (e.g., each ofthe one or more virtual 3D models) may include the definition of ananatomical coordinate system, and the 3D model position may be definedwith respect to the anatomical coordinate system. In certainembodiments, the anatomical coordinate system is defined based onlandmarks (e.g., known anatomical features) that exist in a 3Dstatistical model of the at least one anatomical object used to createthe 3D model, the landmark positions following the shape adjustmentdiscussed with respect to block 306. In certain embodiments, theanatomical coordinate system is defined based on landmarks that areautomatically or manually identified on the 3D model.

The repositioning of block 310 may be performed so that a new positionand orientation of the 3D model correspond to the information indicativeof the normalized projection described in block 304. If the 3D modelincludes multiple anatomy parts, normal articulation of the anatomyparts may be respected during the repositioning. Such normalarticulation may be expressed as explicit rules for repositioning (e.g.rotation around the center of rotation of the femoral head), or as partof the deformable 3D model used to generate the 3D model as discussed atblock 306. It should be noted that in certain aspects block 310 isoptional and the 3D model is not repositioned.

FIG. 6 illustrates an example of the 3D model 500 of FIG. 5repositioned. In particular, the 3D model 500 corresponds to a positionand orientation (e.g., referred to as a normalized position) of the 3Dmodel 500 of the femur with respect to the projection plane of theradiograph 400 adjusted in order to obtain a true AP view. The positionand orientation of the pelvis are maintained. The correction is madewhile respecting the normal articulation of the different anatomy parts.

At block 312, a 2D projection of the 3D model onto a plane (e.g.,projection plane, optimal plane, etc.) is generated. In someembodiments, the 2D projection is generated by image processing module108 automatically tracing, or a user of image processing module 108causing it to trace, rays from a position of a source of x-rays used togenerate the one or more radiographs with respect to the plane alongpoints of the 3D model that are tangential to a surface of the 3D model.The resulting 2D projection may comprise a contour line representation.

FIGS. 7 and 8 illustrate an example of the 2D projection as a contourline representation 700. In particular, the contours of the 3D model 500in the normalized position are projected onto the projection plane ofthe radiograph 400, by tracing rays through the source of the X-rays ofthe radiograph 400 along all surface points in which the surfaces aretangential to the rays resulting in contour representation 700.

In some other embodiments, the 2D projection is generated by imageprocessing module 108 automatically, or a user of image processingmodule 108 causing it to, trace rays from a position of a source ofx-rays used to generate the one or more radiographs with respect to theprojection plane through one or more pixels of the projection plane, andfor each of the one or more pixels, determine a grey value for the pixelbased on a length of a ray associated with the pixel that is includedwithin the 3D model. The resulting 2D projection may be referred to as asynthetic radiograph containing the grey values. In certain aspects, thegrey value for each pixel is further based on an attenuation factor. Incertain embodiments, the attenuation factor is included in thedeformable 3D model used to generate the 3D model as discussed at block306. If the 3D model includes multiple virtual 3D models, theattenuation factor may be different for different 3D models (e.g.,corresponding to different anatomical objects). In certain embodiments,the deformable 3D model includes an attenuation factor that variesthrough 3D space. In certain embodiments, the deformable 3D modelincludes an attenuation factor that varies along the surface normal ofits surfaces. Different approaches to storing/assigning attenuationfactors in a deformable 3D model can be combined.

FIG. 9 illustrates an example of the 2D projection as a syntheticradiograph.

Continuing at block 314, one or more modified radiographs of the atleast one anatomical object are generated based on the 2D projection.The one or more modified radiographs may be generated (e.g.,automatically) by image processing module 108. In certain embodiments,such as if the 2D projection is a contour line representation, the oneor more modified radiographs are generated by identifying one or moreprojected contours of the at least one anatomical object in the 2Dprojection, and morphing the one or more radiographs to align one ormore contours of the at least one anatomical object in the one or moreradiographs to the one or more projected contours. For example, based onthe visual information of the at least one anatomical object, outer andoptionally inner contours of each of the anatomical objects may beidentified in the one or more radiographs. For example, contours of theat least one anatomical object may be identified or detected using knownmethods in the art such as those described in Chen et al. (2014) (Chen,C.; Xie, W.; Franke, J.; Grutzner, P. A.; Nolte, L.-P.; Zheng, G.,“Automatic X-ray landmark detection and shape segmentation viadata-driven joint estimation of image displacements”, Medical ImageAnalysis, April 2014, Vol. 18(3), pp. 487-499). The one or moreradiographs may be morphed (e.g., changed smoothly from one image toanother by small gradual steps using computer animation techniques) sothat the identified contours in the one or more radiographs align with(e.g., best match) the projected contours (e.g., the contour linerepresentation of the 2D projection), such as using image morphingtechniques.

FIG. 10 illustrates an example of the radiograph 400 with contours 1000of the at least one anatomical object identified. FIG. 11 illustrates anexample of the radiograph 400 with both contours 1000 and contourrepresentation 700 overlaid on radiograph 400. FIG. 12 illustrates anexample of a modified radiograph 1200 that is a morph of radiograph 400of FIG. 10 such that the contours 1000 align with the contourrepresentation 700.

In certain embodiments, such as if the 2D projection is a syntheticradiograph, the one or more modified radiographs are generated byperforming non-rigid image registration techniques to register the oneor more radiographs to the 2D projection. The non-rigid imageregistration techniques may be done according to known methods in theart such as those described in Crum et al. (2004) (Crum, W R; Hartkens,T; Hill, D L G, “Non-rigid image registration: theory and practice”, TheBritish journal of radiology, 2004, Vol. 77 Spec No 2, pp. S140-53). Incertain aspects, an original synthetic radiograph may be generated basedon the 3D model before repositioning at block 310. This originalsynthetic radiograph may be registered to the synthetic radiographgenerated at block 312 referred to as a repositioned syntheticradiograph. The transformation for registering the original radiographto the repositioned synthetic radiograph may be stored/recorded, andthen applied to the original one or more radiographs to generate the oneor more modified radiographs.

FIG. 13 illustrates an example of an original synthetic radiograph 1305which is to be registered to repositioned synthetic radiograph 1310.FIG. 14 illustrates an example of an original radiograph 400, and amodified radiograph 1400. Modified radiograph 1400 is generated byapplying the same transformations as used to register original syntheticradiograph 1305 to repositioned synthetic radiograph 1310.

In certain embodiments, the one or more modified radiographs aregenerated by using a 2D transformation (e.g., based on one or morelandmarks of the at least one anatomical object) on the one or moreradiographs to align one or more landmarks of the at least oneanatomical object in the one or more radiographs to the 2D projection.

In certain embodiments, the one or more modified radiographs aregenerated by using a non-uniform scaling on the one or more radiographsto align the at least one anatomical object in the one or moreradiographs to the 2D projection.

Accordingly, process 300 can be performed to generate one or moremodified radiographs in desired positions. FIG. 15 illustrates anexample of a modified radiograph 1500.

Certain embodiments herein further provide systems and methods that useone or more steps of generating one or more modified radiographs. Forexample, certain embodiments further provide systems and methods for 2Dtemplating. 2D templating refers to selecting an appropriate (e.g., mostappropriate) brand, type, shape, and/or size of implant, such as from alibrary of implants (e.g., stored as digital representations on astorage coupled to network 100), by overlaying a two-dimensional medicalimage of an anatomical object with a representation (e.g., a linedrawing, contour drawing, projection of a 3D model, shaded area drawing,etc.) of an implant from the library, which is represented (e.g., drawn)at the correct scale and orientation appropriate for a given view of theanatomy. As described above, the view of the anatomy in a radiograph asachieved by the radiologist might not correspond to the ideal view thatis intrinsic to the 2D line drawings or contours drawings (or othertypes of templates) of implants of the implant library. This might leadto an inappropriate selection of implant.

As discussed, visualization module 112 may be configured to display amodified or enhanced view of anatomy to aid in 2D templating. Forexample, the visualization module 112 may be configured to display on adisplay of computing device 104 a modified radiograph according toprocess 300. In certain embodiments, the normalized projection used inprocess 300 corresponds to the ideal view that is intrinsic to theimplant templates of the implant library. Accordingly, a user can moreaccurately utilize the implant templates on the modified radiograph toselect an appropriate implant. For example, for the 2D templating of thefemoral component of a hip implant, if the femur is externally rotatedin the original radiograph, in certain embodiments, it will be displayedin that radiograph with a reduced neck length. The user might thereforebe inclined to select a small implant. The modified radiograph, incertain embodiments, can display the femur without external rotation andtherefore the correct neck length. The user can therefore have a higherchance of selecting an appropriate-size implant.

In some embodiments, visualization module 112 is configured to display arepositioned synthetic radiograph instead of the modified radiographaccording to process 300. Such a repositioned synthetic radiograph canalso display the anatomy in the normalized projection, but does not havethe full greyscale information that the original radiograph has.However, for 2D templating, it can still lead to more accuratemeasurements and a better implant selection.

In some embodiments, visualization module 112 is configured to displaymodified or synthetic radiographs (e.g., modified radiographs orrepositioned synthetic radiographs or synthetic radiographs that havenot been repositioned) with a visual indication to draw attention to thefact that the user is not looking at the original radiograph but at amodified or synthetic image. This visual indication can be a watermark,a letter, a drawing, an image border, etc., and can be in a contrastingcolor. Alternatively, the visual indication can be a color shift (e.g.whereas the original radiograph is displayed in grayscale colors,modified or synthetic radiographs can be displayed in tones of anothercolor, such as red, green, orange, blue, yellow, pink, cyan, magenta,purple, violet, etc.). Alternatively the grayscale colors can beinverted. Alternatively other known image-processing operations orfilters can be applied as visual indication. Other visual indicationsare possible.

In some embodiments, visualization module 112 is configured to displayat a time a combination of two or more of the original radiograph, oneor more modified radiographs, and one or more synthetic radiographs.Each of the images may be displayed with a visual indication to drawattention to the type of image, and possibly the normalized view that isdisplayed in the image.

In some embodiments, visualization module 112 is configured to displaystandard anatomical measurements on or in the display area surroundingone or more of the original radiograph, a modified radiograph, and asynthetic radiograph. For example, image processing module 108 mayperform blocks 302 and 304 to generate a 3D model of at least oneanatomical object from a radiograph. Image processing module 108 canfurther define one or more landmarks and/or an anatomical coordinatesystem associated with the at least one anatomical object in the 3Dmodel, such as discussed with respect to block 310. Measurement andanalysis module 110 may be configured to perform one or moremeasurements (e.g., standard anatomical measurements) of the at leastone anatomical object in the 3D model based on the one or moreanatomical landmarks or the anatomical coordinate system. Then,visualization module 112 can display the one or more measurements on orin the display area surrounding one or more of the original radiograph,a modified radiograph, and a synthetic radiograph.

Accordingly, measurements can be displayed that are not possible toobtain directly from a 2D medical image, or measurements that cannot beobtained in an accurate way from a 2D medical image, such asmeasurements in directions that are not parallel to the projectionplane.

In some embodiments, the 2D templating is for hip replacement surgery.In such embodiments, the one or more measurements can comprise one ormore of femoral diameter, femoral neck length, femoral offset,acetabulum diameter, acetabulum depth, acetabulum inclination,acetabulum anteversion, pelvic tilt, tilt of the coronal reference planeof the pelvis, pelvic anterior tilt, pelvic lateral tilt, pelvicrotation, neck-shaft angle, femoral torsion (also known as anteversion),femur length, leg length, and/or angles defining the position of thejoint in the image (e.g., adduction/abduction, internal/externalrotation and/or flexion/extension).

In some embodiments, the 2D templating is for knee replacement surgery.In such embodiments, the one or more measurements can comprise one ormore of femoral mediolateral size, femoral anteroposterior size, femoralmedial anteroposterior size, femoral lateral anteroposterior size,tibial mediolateral size, tibial anteroposterior size, tibial medialanteroposterior size, tibial lateral anteroposterior size, varus/valgusangle, femoral shaft angle, and/or angles defining the position of thejoint in the image (e.g., adduction/abduction, internal/externalrotation and/or flexion/extension).

In some embodiments, the 2D templating is for other types of surgery,such as total or reverse shoulder arthroplasty, ankle arthroplasty,wrist arthroplasty, elbow arthroplasty, osteotomies and/or fracturerepair.

In some embodiments, the 3D model and/or deformable 3D model used togenerate the 3D model discussed herein includes normative data, e.g.data relating to healthy anatomy. Accordingly, visualization module 112can not only display anatomical measurements performed on the anatomy ofthe individual patient as discussed, but also values for thosemeasurements as they would be in a healthy situation. For example, inthe case of 2D templating on a hip exhibiting excessive wear of thefemur head, visualization module 112 can display the actual femur necklength, the femur neck length as it ought to be in healthy situation, orboth. In some embodiments, the normative data is obtained by removing adiseased part of the contour of the anatomy from the 2D model and/or 3Dmodel and reconstructing a remainder of the anatomy (e.g., usingstatistical shape modelling (SSM)) to obtain the corresponding health 3Dshape of the anatomy.

In certain embodiments, the visualization module 112 displaysinformation indicative of the one or more anatomical landmarks and/oranatomical coordinate systems on one or more of the original radiograph,a modified radiograph, and a synthetic radiograph. For example, imageprocessing module 108 may perform blocks 302 and 304 to generate a 3Dmodel of at least one anatomical object from a radiograph. Imageprocessing module 108 can further define one or more landmarks and/or ananatomical coordinate system associated with the at least one anatomicalobject in the 3D model, such as discussed with respect to block 310.Then, visualization module 112 can display the one or more anatomicallandmarks and/or anatomical coordinate systems on one or more of theoriginal radiograph, a modified radiograph, and a synthetic radiograph.

Accordingly, anatomical coordinate systems or landmarks can be displayedthat are not possible to obtain directly or accurately from a 2D medicalimage. In some embodiments, the 3D model and/or deformable 3D model usedto generate the 3D model discussed herein includes normative data, e.g.data relating to healthy anatomy. Accordingly, visualization module 112can not only display landmarks and anatomical coordinate systems wherethey are in the individual patient, but also where they ought to be in ahealthy situation. For example, in the case of 2D templating on a hipexhibiting excessive wear of the femur head, visualization module 112can display the actual center of rotation, the center of rotation whereit ought to be in healthy situation, or both.

In some embodiments, visualization module 112 may be configured todisplay a 2D template of an implant on the display in an orientation andposition relative to the position and orientation of the at least oneanatomical object in the one or more modified radiographs.

In some embodiments, visualization module 112 may be configured todisplay modified or enhanced views of implants on an original radiographto aid in 2D templating. For example, visualization module 112 candisplay the original radiograph and a modified radiograph orrepositioned synthetic radiograph side-by-side. The visualization module112 may display the original 2D templates from the implant library onthe modified radiograph or repositioned synthetic radiograph, and anadapted representation of the same 2D template on the originalradiograph. This adapted representation may take the position of theanatomical parts of interest as they are visible in the originalradiograph into account.

For example, the adapted representation of the 2D template may begenerated by image processing module 108 inverting the imagetransformation steps applied to the original radiograph at block 314 togenerate the one or more modified radiographs and apply the invertedimage transformation steps to the original 2D template to generate theadapted representation of the 2D template.

In particular, in certain embodiments, image processing module 108generates the one or more modified radiographs of the at least oneanatomical object based on the 2D projection by applying atransformation to the one or more modified radiographs to align the oneor more modified radiographs with the 2D projection. Accordingly, imageprocessing module 108 applies an inverse of the transformation to a 2Dtemplate of the implant. Further, visualization module 112 displays thetransformed 2D template of the implant along with the one or moreradiographs on a display of the computing device.

In another example, the image processing module 108 can invert therepositioning of the 3D model performed at block 310 and apply it to theoriginal 2D template as placed in the modified radiograph orrepositioned synthetic radiograph. Accordingly, the 2D template can bepositioned in the 3D space of 3D model before repositioning, such as atblock 306. Using information on the position of the source and theprojection plane of the original radiograph and/or information about theregistration and adjustment of the shape of the deformable 3D model asestablished in block 306, the 2D template can then be projected onto theoriginal radiograph.

In particular, in certain embodiments, image processing module 108positions the 3D model with respect to the projection plane by applyinga transformation to the 3D model to align with the desired position andorientation of the at least one anatomical object with respect to theprojection plane. Accordingly, image processing module 108 applies aninverse of the transformation to a 2D template of the implant aspositioned with respect to the one or more modified radiographs togenerate a 3D template of the implant positioned with respect to the 3Dmodel. Further, image processing module 108 generates a second 2Dprojection of the 3D template of the implant onto the projection planein the one or more radiographs. Further, visualization module 112displays the second 2D projection on the one or more radiographs on adisplay of the computing device.

In some embodiments, the library of implants comprises 3D templates ofthe implants. In some such embodiments, image processing module 108 canproject a 3D template onto the original radiograph using the position ofthe 3D model of the at least one anatomical object and possiblyinformation regarding the source and projection plane of the radiograph.In some embodiments, the projection can be derived directly from theregistration and adjustment of the shape of the deformable 3D model asresulting from block 306 without the need for information regarding thesource and projection plane of the radiograph. The 2D template on themodified radiograph or repositioned synthetic radiograph may besimilarly projected based on a 3D template, or, since the modifiedradiograph or repositioned synthetic radiograph corresponds to anormalized view, may come from a 2D depiction of the implant that isstored in the library of implants.

In particular, in certain embodiments, image processing module 108obtains a 3D template of an implant positioned with respect to the 3Dmodel. Further, image processing module 108 generates a second 2Dprojection of the 3D template of the implant onto the projection planein the one or more radiographs. Further, visualization module 112displays the second 2D projection on the one or more radiographs on adisplay of the computing device.

In certain embodiments, the techniques presented herein determine atransformation between a 2D template in a normalized projection and anadapted representation of the implant in the projection of the originalradiograph. The visualization module 112 may display both the originalradiograph and a modified and/or repositioned synthetic radiographside-by-side, or it may merely show the original radiograph, in whichcase the transformation can be computed and applied by image processingmodule 108 automatically (e.g., without notifying the user). Whenvisualization module 112 shows more than one type of radiograph, thetransformation or its inverse may be applied to propagate changes inposition made by the user on one radiograph to the display of the otherradiograph(s). This propagation can be performed in a synchronous(real-time) way or in an asynchronous (automatic update or update ondemand) way, such as by image processing module 108.

In some embodiments, the techniques described here relate to 2Dtemplating for hip replacement surgery. In that case, the adaptedrepresentation of the implant can, for example, relate to an adaptedrepresentation of the acetabular cup component to account for pelvictilt, or to an adapted representation of the femoral component toaccount for an internal or external rotation of the leg.

In some embodiments, the techniques described here relate to 2Dtemplating for knee replacement surgery. In that case, the adaptedrepresentation of the implant can, for example, relate to an adaptedrepresentation of the tibial or femoral component to account for flexionor internal or external rotation of the leg.

Certain embodiments of the disclosure also provide for systems andmethods that allow the user to select certain anatomical parts to beshown or hidden. For example, visualization module 112 may take inputfrom a user of a computing device and display or hide certain anatomicalobjects, parts, etc. For example, for 2D templating, the user istypically interested in one anatomical part at a time. Visualinformation from other anatomical parts may then distract the user orclutter the image. In some embodiments, one or more syntheticradiographs or modified radiographs may be computed as discussed hereinbased only on the virtual 3D model of the anatomical part of interest.In other embodiments, a modified radiograph may be generated in whichall visual information not belonging to the anatomical part of interestis filtered out using image processing functions (e.g., performed byimage processing module 108 and/or visualization module 112).

One advantage of such systems and methods is that the user may only bepresented with the visual information that is relevant to the 2Dtemplating process to be performed.

For example, in the case of hip surgery, the user might be mainlyinterested in 2D templating of the femoral component of ajoint-replacement implant. In that case, the visual information of thepelvis might clutter the image. Certain embodiments of visualizationmodule 112 may then produce and display one or more synthetic and/ormodified radiographs showing only visual information of the femur.

In the case of knee surgery, the user might be mainly interested in 2Dtemplating of the tibial component of a joint-replacement implant. Inthat case, the visual information of the femur might clutter the image.Certain embodiments of visualization module 112 may then produce anddisplay one or more synthetic and/or modified radiographs showing onlyvisual information of the tibia.

In the case of shoulder surgery, the user might be mainly interested in2D templating of the humeral component of a joint-replacement implant.In that case, the visual information of the scapula might clutter theimage. Certain embodiments of visualization module 112 may then produceand display one or more synthetic and/or modified radiographs showingonly visual information of the humerus.

Another advantage is that such systems and methods may generate usefulsynthetic radiographs, such as repositioned synthetic radiographs, ormodified radiographs that are not possible to generate in real life.

For example, in the case of hip surgery, a lateral radiograph is rarelyconsidered useful, as the visual information of the left side of theacetabulum and femur overlay the visual information of the right side ofthe acetabulum and femur. It is almost impossible to discern whichvisual information belongs to which side of the anatomy. However, thesystems and methods provided herein may produce a lateral syntheticradiograph or modified radiograph based solely on one hemipelvis and/orone femur. Such a synthetic radiograph or modified radiograph maypresent the anatomical parts as positioned in the original radiograph(e.g. rotated 90° with respect to a frontal radiograph), or theanatomical parts may be repositioned according to a normalized view asdescribed in block 310.

For example, in the case of knee surgery, a lateral radiograph of thedistal femur may be found confusing, as the visual information of themedial and lateral condyles are overlaid. The systems and methodsprovided herein, such as image processing module 108, may produce asynthetic radiograph based solely on the 3D model of the lateral or themedial condyle.

Certain embodiments herein further provide systems and methods that useone or more steps of generating one or more modified radiographs. Forexample, certain embodiments further provide systems and methods for 3Dtemplating. 3D templating refers to selecting an appropriate (e.g., mostappropriate) brand, type, shape, and/or size of implant, such as from alibrary of implants (e.g., stored as digital representations on astorage coupled to network 100), by comparing and overlaying a 3D modelof the implant (meaning a digital representation of the implant in 3D)from a library of such 3D models on a 3D model of an anatomical objectthat is a digital representation of the anatomical object.

One drawback of surgical templating and planning on 3D models, is that,due to a lack of a visual reference, many users lose their bearings whenviewing the 3D models. Many users, and clinicians in particular,therefore prefer 2D radiographs showing the anatomy in a normalizedprojection, even though much more information can be found in a 3Drepresentation.

Some systems known in the art therefore display a visual referencetogether with the 3D models of the one or more anatomy parts ofinterest. This visual reference can be three or six lines or arrowsrepresenting the 3 axes of a coordinate system, a cube with letters onits faces, three intersecting planes, etc. However, such a visualreference may cause even more confusion if the 3D models of the anatomyparts of interest are not properly aligned with it.

Certain embodiments herein therefore provide systems and methods forimproved 3D templating. In some embodiments, systems and methods areprovided that generate 3D models of at least one anatomical objectaccording to blocks 302 and 306 as described with respect to FIG. 3 .Further, in some embodiments, one or more of these 3D models arerepositioned according to blocks 304 and block 310 as described withrespect to FIG. 3 . In some such embodiments, a normalized projectioncomprises a definition of a desired position and orientation of the atleast one anatomical object relative to a common coordinate system. Theposition and orientation of the at least one anatomical object withinthe common coordinate system can be defined in terms of its anatomicalcoordinate system.

In some embodiments, visualization module 112 displays the 3D models ofthe anatomy parts of interest. For example, visualization module 112displays the positioned 3D model with respect to the projection plane ona display of the computing device. Depending on the type of statisticaldata in the deformable 3D model, the 3D models may contain surface dataand/or grey-value data (e.g., volumetric data), and they may bedisplayed by surface rendering and/or volume rendering respectively. Insome embodiments, visualization module 112 displays a visual referenceindicating the common coordinate system together with the 3D models.

In certain embodiments, one approach to presenting 3D information in away that looks more familiar to the user, is to produce syntheticstacked medical images and to present those to the user in the same wayas CT or MRI images are typically shown. In some embodiments,visualization module 112 displays 2D slices of the positioned 3D model(e.g., corresponding to synthetic stacked medical images).

In certain embodiments, image processing module 108 generates such 2Dslices by performing blocks 302 and 306 of FIG. 3 to create a 3D modelbased on a deformable 3D model of an anatomical object in a radiograph.In some embodiments, the deformable 3D model contains grey-valueinformation (e.g., appearance model, active appearance model, intensitymodel, and/or active shape model) assigned to the nodes of one or morevolume meshes. In some embodiments, the deformable 3D model comprises avolume mesh for each anatomical object and/or the 3D model comprises atleast one virtual 3D model for each anatomical object.

The image processing module 108 then obtains a definition of anormalized pose of the at least one anatomical object (e.g., a desiredposition and orientation) and repositions each of the volume meshesand/or 3D models according to the definition of the normalized pose. Incertain embodiments, the normalized pose is defined in terms of aposition and orientation of each of the at least one anatomical objectwith respect to a common coordinate system. The position and orientationof an anatomical object can be defined by means of its anatomicalcoordinate system.

In some embodiments, the 3D model is displayed by visualization module112 by means of one or more 2D images (e.g., generated by imageprocessing module 108 according to techniques discussed herein)representing slices through the 3D model. In certain embodiments, theslices are made perpendicular to one or more of the axes of the commoncoordinate system. In certain embodiments, a slice is generated bycutting through the 3D model along a plane, and generating a 2D imagecomprising pixels. In certain embodiments, the color of each pixel isdetermined by interpolating between the grey values of the nodes of theone or more volume meshes of the 3D model. Different interpolationmethods may be utilized (e.g. nearest-neighbor interpolation, linearinterpolation, polynomial interpolation, etc.). The slices may begenerated, such as by image processing module 108, as they are displayedby visualization module 112, or alternatively they may be generated onceand stored, such as in image data storage 106. Alternatively, a voxelmesh (e.g., pixel mesh) may be computed (e.g., by image processingmodule 108) wherein the color of each voxel (e.g., pixel) may bedetermined by interpolating between the grey values of the nodes of theone or more volume meshes of the instance of the deformable 3D model. 2Dimages may then be generated (e.g., by image processing module 108) byretrieving one layer of voxels from the voxel mesh.

In certain embodiments, the colors of the pixels and/or voxels may befurther determined by applying one or more filters or image-processingoperations before and/or after the interpolation, such as by imageprocessing module 108. For example, if the deformable 3D model comprisesmore than one volume mesh, the grey values of each of these volumemeshes may be adjusted to different hues. This may, for example, resultin synthetic stacked medical images in which different anatomy parts ofinterest are color coded.

In some embodiments, the 2D images represent coronal, axial and/orsagittal slices through the anatomy. The visualization module 112 maydisplay one or more these at the same time, optionally in combinationwith a 3D view of the 3D model. Some embodiments of visualization module112 may present the user with an interactive way to scroll through theslices.

The methods and systems provided herein may allow the user to performmeasurements in three dimensions and/or to perform 3D templating asdiscussed. In that case the user is able to select the brand, type,shape and/or size of implant that best fits the anatomy of the patientby overlaying 3D models of the implants from an implant library on thedisplayed images of the anatomy. The 3D model of an implant can then besliced (e.g., by image processing module 108) by the plane of an imageand represented by its cross section in that plane (e.g. by displayingthe cross section's contour onto the image, or by overlaying the 2Dshape of the cross section in an opaque or partly transparent color intothe image).

Certain embodiments of the present disclosure provide systems andmethods, such as measurement and analysis module 110, that allow theuser to perform measurements on anatomy in a more accurate way. Thisalso offers the possibility to repeatedly perform the same measurementsin a reliable way.

Certain embodiments of the present disclosure therefore also providesystems and methods, such as measurement and analysis module 110, formaking and comparing measurements at different stages before, duringand/or after a medical intervention. For example, the techniquesdescribed herein can be performed based on 2D radiographs taken before,during, or after a medical intervention, and can therefore be used insystems for pre-operative planning and templating, systems forintra-operative verification or navigation and post-operativeevaluation. All of these can be made to take one or more 2D radiographsas input and facilitate repeatable and reliable measurements.

In some embodiments, measurement and analysis module 110 is configuredto define measurements based on anatomical landmarks, such as thosediscussed herein. For example, a linear measurement can have twoanatomical landmark points as its end points. For example, a diametermeasurement can have a substantially circular or spherical anatomicallandmark as its input. For example, an angular measurement can bedefined using three anatomical landmark points. For example, imageprocessing module 108 may perform blocks 302 and 304 to generate a 3Dmodel of at least one anatomical object from a radiograph. Imageprocessing module 108 can further define one or more landmarks and/or ananatomical coordinate system associated with the at least one anatomicalobject in the 3D model, such as discussed with respect to block 310. Insome embodiments, systems for pre-operative planning or templating,intra-operative verification or navigation and post-operative evaluationmay use such resulting landmarks for the definition of measurements.Alternatively, systems can use landmarks identified on modified orsynthetic radiographs. When the same landmarks are generated before,during, and after the medical intervention, the same measurements can beperformed and compared at these different stages, such as by a user ofor automatically by measurement and analysis module 110.

For example, in certain embodiments, image processing module 108 candefine one or more anatomical landmarks associated with the at least oneanatomical object in the 3D model. Further, measurement and analysismodule 110 can perform one or more measurements of the at least oneanatomical object in the 3D model based on the one or more anatomicallandmarks. Image processing module 108 can identify the one or morelandmarks in one or more additional radiographs. Measurement andanalysis module 110 can perform one or more additional measurements ofthe at least one anatomical object in the one or more additionalradiographs based on the one or more anatomical landmarks.

For example, if the medical intervention is a corrective surgery forfemoroacetabular impingement, the user may pre-operatively measure thelateral center edge angle on a modified AP pelvic radiograph as theangle between a vertical line and a line connecting the center of thefemoral head and the lateral edge of the acetabulum, such as shown inFIG. 16 . To this end, the femoral head and the edge of the acetabulummay be defined as landmarks. This measurement, its definition and thelandmarks that its definition is based on may then be stored, such as instorage 106. Intra-operatively, the same landmarks may be identified ona modified intra-operative radiograph, and a measurement according tothe same definition may be made and compared to the pre-operative value,or used to guide the reshaping of the acetabulum by comparing it to apre-operatively planned target value. Post-operatively, the samelandmarks may be identified on a modified post-operative radiograph, anda measurement according to the same definition may be made and comparedto the pre-operative value or to a pre-operatively planned target value.

It should be noted that in certain embodiments, one or more of themethods described herein is a computer-implemented method. Further,certain steps may be performed automatically, manually by a user of acomputing device, or partially manually and partially automatically suchas based on input from a user of a computing device.

Further, in certain embodiments, a person, such as a clinician,engineer, technician, medical professional, etc., may use a computingdevice to, or the computing device itself may automatically perform oneor more steps of one or more methods described herein.

It is to be understood that any feature described in relation to any oneembodiment may be used alone, or in combination with other featuresdescribed, and may also be used in combination with one or more featuresof any other of the embodiments, or any combination of any other of theembodiments. Furthermore, equivalents and modifications not describedabove may also be employed without departing from the scope of theinvention, which is defined in the accompanying claims.

The methods disclosed herein comprise one or more steps or actions forachieving the described method. The method steps and/or actions may beinterchanged with one another without departing from the scope of theclaims. In other words, unless a specific order of steps or actions isspecified, the order and/or use of specific steps and/or actions may bemodified without departing from the scope of the claims. Further, one ormore blocks/steps may be removed or added.

Various embodiments disclosed herein provide for the use of a computersystem to perform certain features. A skilled artisan will readilyappreciate that these embodiments may be implemented using numerousdifferent types of computing devices, including both general-purposeand/or special-purpose computing system environments or configurations.Examples of well-known computing systems, environments, and/orconfigurations that may be suitable for use in connection with theembodiments set forth above may include, but are not limited to,personal computers, server computers, hand-held or laptop devices,multiprocessor systems, microprocessor-based systems, programmableconsumer electronics, network PCs, minicomputers, mainframe computers,distributed computing environments that include any of the above systemsor devices, and the like. These devices may include stored instructions,which, when executed by a microprocessor in the computing device, causethe computer device to perform specified actions to carry out theinstructions. As used herein, instructions refer to computer-implementedsteps for processing information in the system. Instructions can beimplemented in software, firmware or hardware and include any type ofprogrammed step undertaken by components of the system.

A microprocessor may be any conventional general-purpose single- ormulti-chip microprocessor such as a Pentium® processor, a Pentium® Proprocessor, a 8051 processor, a MIPS® processor, a Power PC® processor,or an Alpha® processor. In addition, the microprocessor may be anyconventional special-purpose microprocessor such as a digital signalprocessor or a graphics processor. The microprocessor typically hasconventional address lines, conventional data lines, and one or moreconventional control lines.

Aspects and embodiments of the inventions disclosed herein may beimplemented as a method, apparatus or article of manufacture usingstandard programming or engineering techniques to produce software,firmware, hardware, or any combination thereof. The term “article ofmanufacture” as used herein refers to code or logic implemented inhardware or non-transitory computer readable media such as opticalstorage devices, and volatile or non-volatile memory devices ortransitory computer readable media such as signals, carrier waves, etc.Such hardware may include, but is not limited to, field programmablegate arrays (FPGAs), application-specific integrated circuits (ASICs),complex programmable logic devices (CPLDs), programmable logic arrays(PLAs), microprocessors, or other similar processing devices.

1. A computer-implemented method of generating a computer-basedradiographic representation of at least part of one anatomical object,the method comprising: obtaining, at a computing device, one or moreradiographs of at least part of one anatomical object, each of the oneor more radiographs comprising a 2D visual representation of the atleast part of one anatomical object in a corresponding projection plane;obtaining, by the computing device, information indicative of a desiredposition and orientation of the at least part of one anatomical objectwith respect to a first projection plane; corresponding to a firstradiograph of the one or more radiographs; generating, by the computingdevice, a 3D model of the at least part of one anatomical object basedon the one or more radiographs; repositioning, by the computing device,the 3D model based on the information indicative of the desired positionand orientation of the at least one anatomical object with respect tothe first projection plane; generating, by the computing device, a 2Dprojection of the 3D model onto the first projection plane, thegenerating comprising tracing rays, from a position associated with asource of x-rays used to generate the first radiograph, along points onthe 3D model where the 3D model is tangential to the traced rays,wherein the 2D projection comprises a contour line representation; andmodifying by the computing device, at least one of the one or moreradiographs of the at least part of one anatomical object based on the2D projection- to generate at least one modified radiograph, themodifying comprising: morphing the at least one of the one or moreradiographs to align one or more contours of the at least part of oneanatomical object in the at least one of the one or more radiographs tothe contour line representation.
 2. The method of claim 1, wherein thefirst radiograph further comprises position information of the positionassociated with the source of x-rays used to generate the firstradiograph.
 3. The method of claim 1, wherein generating the 3D modelcomprises adjusting a shape of the 3D model to align with a shape of theat least part of one anatomical object in the one or more radiographs.4. The method of claim 1, wherein the 3D model includes statisticalinformation about the at least part of one anatomical object, thestatistical information comprising one or more of a statistical shapemodel, an articulated statistical shape model, an active shape model, anappearance model, and an intensity model.
 5. The method of claim 1,further comprising: displaying the at least one modified radiograph on adisplay of the computing device.
 6. The method of claim 5, furthercomprising: displaying a visual indication to indicate that the at leastone modified radiograph has been modified.
 7. The method of claim 5,further comprising: defining one or more anatomical landmarks associatedwith the at least part of one anatomical object in the 3D model;performing one or more measurements of the at least part of oneanatomical object in one or more of the 3D model or the at least onemodified radiograph based on the one or more anatomical landmarks; anddisplaying the one or more measurements on the display.
 8. The method ofclaim 7, wherein the one anatomical object is a hip of a patient, andwherein the one or more measurements comprise one or more of a femoraldiameter, a femoral neck length, a femoral offset, an acetabulumdiameter, an acetabulum depth, an acetabulum inclination, an acetabulumanteversion, a pelvic tilt, a tilt of the coronal reference plane of thepelvis, a pelvic anterior tilt, a pelvic lateral tilt, a pelvicrotation, a neck-shaft angle, a femoral torsion, a femur length, a leglength, an adduction or abduction, an internal or external rotation, ora flexion or extension.
 9. The method of claim 7, wherein the oneanatomical object is a knee of a patient, and wherein the one or moremeasurements comprise one or more of a femoral mediolateral size, afemoral anteroposterior size, a femoral medial anteroposterior size, afemoral lateral anteroposterior size, a tibial mediolateral size, atibial anteroposterior size, a tibial medial anteroposterior size, atibial lateral anteroposterior size, a varus or valgus angle, a femoralshaft angle, an adduction or abduction, an internal or externalrotation, or a flexion or extension.
 10. The method of claim 5, furthercomprising: defining one or more anatomical landmarks associated withthe at least part of one anatomical object in the 3D model, whereindisplaying the one or more of the modified radiographs comprisesdisplaying information indicative of the one or more anatomicallandmarks.
 11. The method of claim 5, further comprising: displaying a2D template of an implant on the display.
 12. The method of claim 1,further comprising: applying an inverse of the morphing to a 2D templateof an implant; and displaying the morphed 2D template of the implantalong with the at least one of the one or more radiographs on a displayof the computing device.
 13. The method of claim 1, whereinrepositioning the 3D model with respect to the first projection planecomprises applying a transformation to the 3D model to align the 3Dmodel with the desired position and orientation, and further comprising:applying an inverse of the transformation to a 2D template of an implantto generate a 3D template of the implant; generating, by the computingdevice, a second 2D projection of the 3D template of the implant ontothe first projection plane in the first radiograph; and displaying thesecond 2D projection on the first radiograph on a display of thecomputing device.
 14. The method of claim 1, further comprising:obtaining a 3D template of an implant; generating, by the computingdevice, a second 2D projection of the 3D template of the implant ontothe first projection plane in the first radiograph; and displaying thesecond 2D projection on the first radiograph on a display of thecomputing device.
 15. The method of claim 1, further comprisingdisplaying the repositioned 3D model on a display of the computingdevice.
 16. The method of claim 15, wherein the displaying comprisesdisplaying 2D slices of the repositioned 3D model.
 17. The method ofclaim 1, further comprising: defining one or more anatomical landmarksassociated with the at least part of one anatomical object in the 3Dmodel; performing one or more measurements of the at least part of oneanatomical object in the 3D model based on the one or more anatomicallandmarks; identifying the one or more anatomical landmarks in one ormore additional radiographs; and performing one or more additionalmeasurements of the at least part of one anatomical object in the one ormore additional radiographs based on the one or more anatomicallandmarks.
 18. A non-transitory computer-readable medium havingcomputer-executable instructions stored thereon, which, when executed bya computing device, cause the computing device to perform a method ofgenerating a computer-based radiographic representation of at least partof one anatomical object, the method comprising: obtaining one or moreradiographs of at least part of one anatomical object, each of the oneor more radiographs comprising a 2D visual representation of the atleast part of one anatomical object in a corresponding projection plane;obtaining information indicative of a desired position and orientationof the at least part of one anatomical object with respect to a firstprojection plane corresponding to a first radiograph of the one or moreradiographs; generating a 3D model of the at least part of oneanatomical object based on the one or more radiographs; repositioningthe 3D model based on the information indicative of the desired positionand orientation of the at least one anatomical object with respect tothe first projection plane; generating a 2D projection of the 3D modelonto the first projection plane, the generating comprising tracing rays,from a position associated with a source of x-rays used to generate thefirst radiograph, along points on the 3D model where the 3D model istangential to the traced rays, wherein the 2D projection comprises acontour line representation; and modifying at least one of the one ormore radiographs of the at least part of one anatomical object based onthe 2D projection to generate at least one modified radiograph, themodifying comprising: morphing the at least one of the one or moreradiographs to align one or more contours of the at least part of oneanatomical object in the at least one of the one or more radiographs tothe contour line representation.
 19. A computing device comprising: amemory; and a processor configured to perform a method of generating acomputer-based radiographic representation of at least part of oneanatomical object, the method comprising: obtaining one or moreradiographs of at least part of one anatomical object, each of the oneor more radiographs comprising a 2D visual representation of the atleast part of one anatomical object in a corresponding projection plane;obtaining information indicative of a desired position and orientationof the at least part of one anatomical object with respect to a firstprojection plane corresponding to a first radiograph of the one or moreradiographs; generating a 3D model of the at least part of oneanatomical object based on the one or more radiographs; repositioningthe 3D model based on the information indicative of the desired positionand orientation of the at least one anatomical object with respect tothe first projection plane; generating a 2D projection of the 3D modelonto the first projection plane, the generating comprising tracing rays,from a position associated with a source of x-rays used to generate thefirst radiograph, along points on the 3D model where the 3D model istangential to the traced rays, wherein the 2D projection comprises acontour line representation; and modifying at least one of the one ormore radiographs of the at least part of one anatomical object based onthe 2D projection to generate at least one modified radiograph, themodifying comprising: morphing the at least one of the one or moreradiographs to align one or more contours of the at least part of oneanatomical object in the at least one of the one or more radiographs tothe contour line representation.